home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
The PC-SIG Library 9
/
The PC-SIG Library on CD ROM - Ninth Edition.iso
/
1601_700
/
DISK1617
/
DISK1617.ZIP
/
DSMLOGIC.DOC
< prev
next >
Wrap
Text File
|
1990-04-20
|
35KB
|
824 lines
DISAM 3 LOGIC
Version 3.5
Apr. 20, 1990
Written by:
Robert Pearce
2325 W. Cabana
Mesa, AZ. 85202
(602) 835-9189
FILE CONTROL BLOCKS
When a DISAM file is defined, the file is composed of
three basic blocks. 1)Control block, 2)Index block, and
3)Data block.
CONTROL BLOCK
The control block is 128 bytes long and is used to
track some of the statistical data about the file's perfor-
mance and to hold the address constants (ADCONS) of the
buffers since more than one file can be accessed at a time.
See DSMADCON.DEF for the specific fields defined. Some are
defined here.
offset label value (all values are in decimal)
000 CBS 128 Control Block Size
002 IBS U/D Index Block Size Min size=(FKL+4)*4+8
004 DBS U/D Data Block Size n*IBS
006 FKL U/D File Key Length (user defined)
008 FKO U/D File Key Offset (user defined)
010 NFB 0 Next File Block (to be assigned)
012 IRC 0 Index Record Counter
014 DRC 0 Data Record Counter
016 ISC 0 Index Split Counter
018 DSC 0 Data Split Counter
020 FS1 0 File Status Flag1 EQU
FOF 00000001B File Open Flag 01
FUF 00000010B File Update Flag 02
IUF 00000100B Immed Update Flag 04
QBOF 00001000B Quick-BASIC Open Flag 08
021 FS2 0 File Status Flag2
FSO 00000001B File Share Option 01
SIF 00000010B Secondary Index Flag 02
022 PFS 0 Percent Load Free Space
024 SIKL 0 Secondary Index Key Length
026 SIKO 0 Secondary Index Key Offset
028 HANDLE0 DISAM File Handle
The following are address constants for file buffers
030 IBA 0 Index Buffer Address
032 IFSA 0 Index Free Space Address
034 INBA 0 Index Next Block Address
036 DBA 0 Data Buffer Address
038 DFSA 0 Data Free Space Address
040 DNBA 0 Data Next Block Address
The following ADCONS point file records.
042 CIRA 0 Current Index Record Address
044 NIRA 0 Next Index Record Address
046 CDRA 0 Current Data Record Address
048 NDRA 0 Next Data Record Address
The following ADCONS point file blocks.
050 CIBN 0 Current Index Block Number
052 NIBN 0 Next Index Block Number
054 CDBN 0 Current Data Block Number
056 NDBN 0 Next Data Block Number
The following locations are used during a block split
058 SRL 0 Save Record Length
050 CBN 0 Changed Block Number
062 UBUFA 0 User Buffer Address
The following ADCONS point secondary index records.
064 CSIRA 0 Current Secondary Index Record Address
066 NSIRA 0 Next Secondary Index Record Address
068 CSIBN 0 Current Secondary Index Block Number
070 NSIBN 0 Next Secondary Index Block Number
Offsets 072 through 126 are not used.
INDEX BLOCK
The index block (physical block 0) is IBS long, as
defined by the user at definition time. In number terms this
means that the minimum size is: (Key-length+4)*4+8 or 512
bytes, the default, or what ever the user specified. It
contains fixed length records equal to the key length plus
four bytes. Two used to specify the record length and two
used to point to the data block.
----------------------------------------------------------
|IRL| key |DBN|IRL|X'FFFFFFFFFF'|DBN|000| |
|---------------------------------------------- |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| ------------|
| | IFS | NIB |
----------------------------------------------------------
|___2___|___________FKL___________________|___2__|
IRL record key DBN
The length of the record key is defined by the user.
Maximum current key length is 125 bytes.
Four bytes at the end of the index block are used for
block control.
IFS is the free space in the block. This is initially
set to IBS-8
NIB is the pointer to the next index block. The last
index block points to zero.
IBS = index block size
IRL = index record length
DBN = data block number
IFS = index free space
INB = index next block
Based on a 512 byte index block:
offset label = value
000 start of index block
508 IFS = 504=(IBS-8)
510 INB = 0
SECONDARY INDEX BLOCK
The secondary index block when used is IBS long, the
same as the index block. It is placed at the end of the file
and the file is marked as read only. The secondary index
provides an alternate way to find a set of data records.
----------------------------------------------------------
|SIRL| SI key |DBN|CDRA|SIRL|X'FFFFF'|DBN|CDRA|000| |
|----------------------------------------------------- |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| ------------|
| | IFS | NIB |
----------------------------------------------------------
|___2___|__________SIKL___________|___2___|___2___|
SIRL SI record key DBN DRO
The length of the Secondary Index key is defined by
the user. Maximum current key length is the File Key Length
(FKL) times 2.
Four bytes at the end of the secondary index block are
used for block control.
IFS is the free space in the block. This is initially
set to IBS-8
NIB is the pointer to the next index block. The last
index block points to zero.
SIRL = secondary index record length
DBN = data block number
DRO = data record offset
IFS = index free space
INB = index next block
Based on a 512 byte index block:
offset label = value
000 start of index block
508 IFS = 504=(IBS-8)
510 INB = 0
DATA BLOCK
The data block, (physical block 1) is constructed in
the same way as the index block but holds the data records.
The size, (DBS) is defined by the user at file definition
time. It must be a multiple of the index block size. Minimum
size is equal to the index block. Max can be up to 10,000
bytes. 2048 is the default size.
----------------------------------------------------------
|DRL| data record |DRL|X'FFFFFFFFFF'|000| |
|------------------------------------------- |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| ------------|
| | DFS | NDB |
----------------------------------------------------------
|__2__|_______________Variaable_____________________|
DRL data record
DBS = data block size
DFS = data free space (DBS-4)
DNB = data next block (DBS-2)
DRL = data record length
The record length value does not include the 2 length bytes.
Based on a 2048 byte data block:
offset label = value
0000 start of data block
2044 DFS = 2040=(DBS-8)
2046 DNB = 0
The "8" in the initial free space calculation allows 4
control bytes, 2 terminator bytes and 2 extra slop bytes.
FILE DEFINITION
At file definition time the user provides the key
length (FKL) and the key offset (FKO). The key length is
edited for less than 126 bytes and not zero. The offset is
edited for less than 255-FKL.
The Index and the data block sizes, (IBS and DBS,) are
also provided by the user with some restrictions. The index
block must be at least 4 times the key length. Specifically,
the minimum size is FKL+4 times 4 plus 8. The data block
must be a multiple of the index block size. There are no
maximum sizes. However current design limits are 3K per DFH3
buffer and 5 buffers max. This allows the work area for 1
file to be 15K in buffer space. Or 2 files of 7.5K etc.
Beyond this you will get a MS-DOS memory allocation error
and be forced to reboot the system.
Load free space, (PFS), is the last value required
from the user. This is used only when the DISAM file is
loaded using the DISAM utility LOAd function. This
percentage is subtracted from the data free space value
before the data block is filled. When the free space value
is used up, the percentage is restored and the block is
written. In effect, this maintains a percentage of free
space in each data block. e.i. A 1000 byte data block with
20 percent free space is loaded with 800 bytes worth of data
records. the 20 percent is then restored (200+) to the data
free space and the block is written. This will allow a 20
percent growth in each data block before a data split is
required.
These five values are placed in the control block as
FKL, FKO, IBS, DBS, and PFS.
Based on the FKL a record of high-values is placed in
the index block with a data block pointer of 1.
Based on FKL and FKO a record of high-values is placed
in the data block. The record lengths are subtracted from
the free space fields respectively and all three blocks are
written to the DISAM file.
These are known as logical end of file records. They force
all records to be inserted below them.
FILE HANDLER EXECUTION
DHF3.COM is executed by the user to load the DISAM
file handler into the system. The file handler checks to see
if it has already been loaded by checking the entry point
address (12:0). It expects to find a four byte
segment:offset address pattern. A compare of three sets of
addresses at 12:0, 12:4 and 12:8 is done. If these addresses
are all the same they are considered as trap addresses and
the load continues. DFH3 builds a long JMP instruction (5
bytes) at that address to its own entry point. Based on the
number of buffers specified, default=1, it acquires 3K
chunks of memory. Then it terminates and stays resident.
If the address is not part of a segment:address train
then DFH3 checks for a x'EA' at 12:0. If this is true then
DFH3 displays an already loaded message and exits. If there
is any other value, DFH3 assumes that the address is being
used by another interrupt and displays a message to that
effect and terminates. To change the location of the entry
address will require a reassembly of DFH3.COM or ZAP
DFH3.Z01.
Once loaded, DFH3 becomes a FAR subroutine to the
calling program. The user entry address, 12:0, is used
because MS-DOS will load DFH3 where ever there is space and
the user has no control over its placement. In the GWBASIC
program the DFH3 segment address is coded DEF SEG=&H12 and
the offset is coded DFH3=&h0. This code sets up the entry
address for the file handler at 12:0.
FILE ACCESS (PASSING VARIABLES)
There are two variables normally sent to DISAM. Some
functions require only one, e.i. close. But because a
return-code is returned in the second variable, it is still
required with a minimum length of one byte. The first
variable is the function code and the buffer number
separated by a comma. If there is no comma and buffer
number, "1" is assumed. The second variable will depend on
the first. Usually the second will be a file name, a data
record, or a record key.
The variables' string descriptor blocks are passed on
the stack to DFH3. Only the CS register points to the
subroutine's base address. The stack belongs to GWBASIC as
does the DS register. DFH3 tries to use as little of the
stack as possible due to not knowing how deep the subroutine
is into the stack.
The variables are string descriptor addresses offset
DS. The DS register is saved and the strings are moved into
DFH3's address space (CS). All work done by DFH3 will be
done in it own address space so that no problems will be
created with GWBASIC.
The first variable, the file function, is converted to
upper case and tested. Next the buffer number is checked.
The buffer specified is an offset into the buffer address
table. It's value may be from 1 to 5. The number is doubled
and the table accessed. Functional chart follows.
Initial Buffer status
| closed | open | invalid |
------------------------------------------
Function: O,Q | open * | error | error |
C | ignore | close | ignore |
All others | error | open *| error |
------------------------------------------
* The address of the buffer is placed in the DS register.
In the case of the close, the buffer address is closed after
being moved to the DS register.
The function of opening and closing buffers is as
follows. Buffer addresses come in three flavors. Open,
Closed and nonexistant. Nonexistant buffer addresses are the
buffers not specified. e.i. If you specify 3 buffers, then 4
and 5 are nonexistant. Thus, zero. An open buffer address
is a valid address pointing to a 3K block of memory. A
closed buffer address is a valid buffer address or'd with
X'8000'. 2CF0 is valid and open. ACF0 is valid and closed
The second variable is checked for a minimum length of
one (1) byte. If not, control is passed immediately back to
the calling program. Exit.
Next, the users data is moved into DFH3's address
space. The function code is used to determine what is to be
done.
When the function is completed the return-code or
record is moved to GWBASIC's address space defined by the
second variable. Normally the length code in the string
descriptor is shortened to fit the response. If there is not
enough length to hold the record, the record is truncated.
If the file was opened as a Quick-BASIC file the length is
NOT changed and it is up to calling program to determine how
much of the returning string is valid.
FILE ACCESS (OPEN)
The file open can originate from one of two places.
The first is a normal open. The second is a Quick-BASIC
open. The difference is that the Quick-BASIC open sets the
QBOF flag.
The open process checks to see if an open has already
been processed.(This is not the same as assigning a buffer
address.) If the file is already open, an invalid function
"9" is returned to the calling program.
There is another user error condition, "8", which is
caused when the calling program ABnormally ENDs (ABENDs)
while the DISAM file is open. MS-DOS will close the file at
the time of the abend. However the open flag, FOF, and
possibly the update flag, FUF, May still be set. If the FUF
flag is set then the update data is lost. The open will
reset the flags and return to the calling program with an
(8) return-code.
If the file is not found then the user will get a "7"
(file not found) return-code.
Next the control block is read to get the sizes of the
index and data blocks.Using the buffer base address the
index block and data block ADCONS are built.
A check for an open file is again done. This time with
the control block gotten from the file. If the file is
closed, then the open continues. If the file is already
marked as open, the share option is checked. If the share
option is "Y" (1), then the open continues. If not, then
the file is closed and the buffer is released. An error of
"5" (sharing not allowed) is returned to the user.
Continuation of the open function includes marking the file
as open and writing the control block back to disk. This
allows other programs to check to see if the file is open.
CIBN is set -1 and NIBN is set to 0. The Read Index
Block (RIB) subroutine is called which forces a read and the
index buffer is filled. CDBN is set to 0 and NDBN is set to
1. a call to Read Data Block (RDB) forces a read and the
data buffer is filled. Last the file is marked open (FOF),
the return code is set to "0" and control is returned to the
calling program.
FILE ACCESS (CLOSE)
The close routine first resets the Immediate Update
Flag (IUF). Then checks the File Status Flag (FSF)to see
if the file has been opened. If not the routine terminates
without error.
The File Update Flag (FUF)is checked. If it is set,
which means there is still an unwritten buffer, the data
buffer is written.
The "update" and "file open" flags are reset and the
control buffer is written.
The file is then closed to MS-DOS and routine
terminates without error.
RECORD KEY SEARCHES
When I initially designed DISAM for the HDOS 2.0
environment, module size was more important than access
speed. So I chose not to use rotating index-block searches.
I continued this into the MS-DOS environment.
All key searches start at index block zero.
When a search for a key is initiated, the index buffer
is checked for the first index block. If it is not there it
is read in from the file.
The index search looks for a key equal or greater than
the user's key. The index block chain is used to search
through multiple index blocks until the index key conditions
are met.
The data block number associated with the index key is
used to read the data block.
Before the data block is read into the buffer a check
is made to see if the data block in the buffer is the one
wanted or has been updated (FUF). If the data block is
already in the buffer, the search for the record begins. If
the data buffer is not the correct one and has been updated,
it is written to the file. The data block wanted is read
into the buffer and the search for the data record begins.
Each data record is checked sequentially against the
user's key. As the search progresses, data record address
fields are maintained. The CDRA (current data record
address) points to the record under test. The NDRA (next
data record address) points the next data record. When the
match is found, control is returned to the calling function
with the carry flag clear. If there is no match, control is
returned with the carry flag set. Since the data records are
stored in ascending sequence, a data key greater than the
user key is a not found condition.
With all this in mind, we move on to the rest of the
file functions.
FILE ACCESS (ADD)
A search for the record is initiated. If the record is
found, an error is returned to the calling program.
The add function first checks to see if there is
enough free space in the data block for another record.
If there is, the free space value is reduced by the
length of the record plus 2. (2 bytes for the length value)
The CDRA is then used as the data block dividing point. All
data records fron the CDRA to the end of the data in the
block are moved up for a distance of the new record length
plus 2. The new record is then inserted in the space
provided by the move. Control is passed back to the calling
program.
If there is not enough space, the block is split and
then the record is added as above.
Part of the Add routine is used by the "PUT" logic and
is explained there.
FILE ACCESS (DELETE)
A search is initiated for the record. If the record is
not found, an error is passed to the calling program.
If it is, the record length plus 2 is added to the
free space value. All records beyond the record to be
deleted (NDRA) are moved down in the block to cover the
deleted record(CDRA). Control is passed back to the
calling program.
Part of this function is used by the "PUT" function.
FILE ACCESS (GET)
The GET access can be performed in three ways.
1) A sequential read.
(First character in the key is a space)
2) A partial (generic) key read.
(The user key length is less than FKL)
3) A full key read.
(The user key is equal or greater than FKL)
For a sequential read, the NDRA is moved to the CDRA
and the data record is gotten from the CDRA.In the event
that the NDRA is zero, (end of data block), the NDB is used
to get the next data block. If the NDB is zero then the
logical end of file has been reached.
For a generic read, a key compare is done only for the
length of the user key.
For a full read, a key compare is done for the length
of the file key.
In both keyed reads a search is initiated for the
record. If the record is not found, an error is passed to
the calling program.
Using the CDRA, the record is placed in the address
space defined by the second calling parm not to exceed the
input parm length. In a GWBASIC call to DFH3 it is
important that the length of the second parm be 255 bytes
insure that the whole record is returned.
If the file record is less that the provided length,
the length is shortened to the file record length.
The exception is for the "Q" open for Quick-BASIC. The
user's length in the second field is used regardless of data
record length. It is up to the Quick-BASIC program to know
how much of the return field to use. Quick-BASIC will get a
string integraty violation if the string descriptor length
is changed outside of Quick-BASIC.
FILE ACCESS (PUT)
A search is initiated for the record using the full
key. If the record is not found, an error is passed to the
calling program.
The PUT function replaces an existing record using the
DELETE and ADD functions. A delete for the record is issued
using the DELETE routine. Then an add for the record is
issued using the ADD routine. In this way record length
changes are allowed. If the replacement record causes a data
block split it is handled as a normal add.
SECONDARY INDEXING
Once a DISAM file has been defined and loaded it may
have a secondary index created. The "Sec" finction of DISAM
will add the secondary index to the end of the file. This
will also lock the file as READ ONLY. You may reorganize the
file and that will allow it to be a normal DISAM file. The
secondary index records point directily to the physical
location of the data records on a 1-for-1 basis. Any changes
to the file will render the secondary index useless.
SPLIT LOGIC
When a record is to be added to a block, first the
record length plus 2 is subtracted from the free space
field. If this subtraction causes an overflow, carry flag to
be set, there is not enough space in the block for the
record.
When a block is split approximatly half of the data is
copied to another block and half of the data is kept in the
current block. When a DATA block is split, a new index
record is created. So before we can split a DATA block
there must be enough space in the index block for another
record.So the index block must be checked to see if there
is enough space for another key record. If there is, the
data block split continues. If not, the index block is split
first.
INDEX SPLIT
The index block is split by dividing the block size by
2 and stepping through the block, subtracting the record
length until the value goes negative. At this point the
address of the next record is saved (NIRA) and the index
string is terminated. The free space is recalculated. The
block chain is updated to point to the Next File Block
(NFB). The index record is written.
Using the same data in the index buffer, the upper
index records are moved to the bottom of the index block.
The free space is recalculated and the new index record is
appended to the file using the next file block number. Last
the Next File Block pointer in incriminated and the control
record is written.
At this point we are abend safe. Control is passed
back to the "ADD" function for another try.
DATA SPLIT
The data split occurs in the same way as the index
split with three exceptions.
1) The original data block index record is updated
with the NFB value.
2) The key of the last record in the lower data buffer
is written to the index buffer.
3) The Next Block Number is calculated based on the
algorythm DBS/IBS.
All buffers are written after a data block split to
protect the file. Control is passed to the "ADD" function.
EXAMPLE:
1) Before DISAM file image.
0 index
--------------
| ee1 kk2 oo3|
| FF4 |
| 0|
--------------
1 data 2 data 3 data 4 data
------------- ----------- ------------ -----------
|aaaaaaaaacc| |ggggggiii| |mmmmmmmmmm| |qqqqqqqqq|
|cccccccccee| |iiiiiiikk| |mmmmmmmmoo| |qqssssssu|
|eeeeeeee 2| |kkkkkk 3| |ooooooo 4| |uuuFF 0|
------------- ----------- ------------ -----------
2) "jjjjjjjjjjj" record added, causes data block split
0 index
--------------
| ee1 kk2 oo3|
| FF4 |
| 0|
--------------
1 data 2 data 3 data 4 data
------------- ----------/ ------------- -------------
|No | |ggggggiii| |No | |No |
| Change | |iiiiiiikk| | Change | | Change |
| 2| |kkkkkk 3| | 4 | | 0|
------------- --/-------- ------------- -------------
data split
3) data block splits
0 index
--------------
| ee1 ii2 kk5|
| oo3 FF4 |
| 0|
--------------
1 data 2 data 3 data 4 data 5 data
------------- ----------- ---------- ---------- ----------
|No | |ggggggiii| |No | |No | |kkkkkkk |
| Change | |iiiiiii | | Change | | Change | | |
| 2| | 5| | 4| | 0| | 3|
------------- ----------- ---------- ---------- ----------
4) "jjjjjjjjjjj" record added
0 index
--------------
| ee1 ii2 kk5|
| oo3 FF4 |
| 0|
--------------
1 data 2 data 3 data 4 data 5 data
------------- ----------- ---------- ---------- ----------
|aaaaaaaaacc| |ggggggiii| |mmmmmmmm| |qqqqqqqq| |jjjjjjjj|
|cccccccccee| |iiiiiii | |mmmmmmoo| |qqsssssu| |kkkkkk |
|eeeeeeee 2| | 5| |ooooo 4| |uuuFF 0| | 3|
------------- ----------- ---------- ---------- ----------
